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ABSTRACT 

An index that reJlects the accuracy of selection 
aesociated with a predictive validity of "r" is presented. Based on 
dheppard's thoerein on median dichotomies, it is a measure of 
improvement over chance assignment to "accept" (or "reject") . Eecause 
the index is a measure of the accuracy of this assignment^ rather 
than of variation from prediction throughout the distribution, the 
index is deeined to be a more appropriate measure than r squared or 
indices based on r squared when the purpose of testing is selection 
or placements (Author/DB) 
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Abstract 



A indejf that reflacts the accuracy of selection associated with a pre- 
dictive validity of £ is presented. MbbA on Sheppard's theorem on nedlao 
dichotomies, It is a mtaaure of imprcvemeat over chance assignment to 
"aeeept" (or "rejeet"). Beeauie the index Is a measure of the aceuracy 
of this asilgnment, rather than of variation from prediction throughout 

the distribution, the index Is deemed to be a more appropriate measure 

2 2 
tton r or iBdloea baaed on r when the purpose of testing is Beleetion 

or placement. 



Merely knowing that the correlation between a sat of test score i and 
score criterion measure it different from zero (at some level of statistical 
significance) is not enough for the evaluation of the coeffieient. For a 
-mlidity coefficient to be Judged properly, one must "evaluate the size of 
the correlation in the light of the uset to be made of the test" (Anaitasi, 
1968, 130). Such evaluations appear to have traditionally taken one of 
two directions. The first will be deemed the "varlance-accounted-fDr" 
approach, which haa concentrated on r^, the squared correlatlcn between the 
predictor (test) and criterion (variable),^ The second, called the ^'declsinn- 
theory" approach, is concerned with the acoui-acy of Identification of positive 
and ne^tive instances of erlterlon status such identification being on the 
basis of one's predictor status. The flrit has resulted in the derivation of 
a number of indices for interpreting eosrolatiofi coefficients, which the 
second sees as generally being too "restrictive J* 

Variance -aeeounted-f or 

2 

r , the "coefficient of determination" or "coefficient of associction" Is 
the proportion of ■mriance on the criterion mrlable aeeounted for by variance 
on the predletor. It may ba demonstrated algebraically that r^ is the ratio 
of the variance of the predicted criterion scores to the variance of the ob- 
tained criterion scores. The complement of r^, 1. i - r^, has also been 
deemed to be useful. Known as the "coefficient of nondetermination" or •'co- 
efficient it nonasBociation/' it is obvioualy the proportion of variance on 
the criterion not accounted for 'by that on the predictor. Consideration of 
k can lead to such statements as, "Since most correlatlonB between tests of 
academic aptitude and college frsshman grade point averages are in the order 
of ,k, we can say that after sixty years of the mental measurement movement, 
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we Imve improved our techno losr to the point where only 85 percent of out 
prediction is error.'' 

2 

The square root of If is of some interest also. Known as the "coefficient 
of alienation/* it serves as the factor by ^hich the standard deviation of the 
criterion is multiplied In order to produce the standard error of estimate. 
Of more immediate concern ia the uie of k in the computation of E, Hull's 
(19£7) "Index of forecaating efficiency," E ^ r^ and is usually 

interpreted as representing the proportion of improvement over chance pro^ 
vided by a given validity coefficient* 

The obvious logic of E and its consermtive evaluation of various values 
of r have posed a dilemma for psychometriclans for more tten t^o score yearn* 
Ttet E m ,i%k when r m ,5 i^hen verballied as representing a ''I3,k percent 
Improvement over chance is certainly irksome and poisibly unreasonable. 
To produce an E of .5 raqulree that the criterion^related validity of a test 
be greater than ,8S, a value that could protebly never be attained by any 
mental teat, 

Apologlits have^ for years^ made attempts to rfcopclle thii dilemma. 
As an example^ conilder Guilford *s statement i 

Batter teets^ with mlidlty coefficienti of .60, have an E of 
about 20 per cent| and itill better teats^ when r m ^75^ have 
an E of about 3I+ percent^ Although these efflaieneles may also 
seem small, we must treat them in a re]jitive, not an absolute 
sense* It is prdbable that the efficiency of predictions ta.sed 
upon the average unsysteflmtie interview is less than 5 percent, 
With*asas a tmse, the efficiency of tests looks much better, 
(1965| p. 378)* 

A more fruitful means of reconeillng the conserTOtlsm of B with the 
demonstrable mlm of tests is by considering Just what it is that Is being 
improved over elmnce. wtmt is being Ifflproved is the variation of obtained 
icoree about the recession line estiaating them# 
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Decision Theory 

The declsion-thaoratic appproach looks not at variation of obtained scores 
about a regression line but merely at correctness of classification. While 
there are nmny proponents of decision theory working in ntiny areas of 
statletiGS^ moit are overlooked here to concentrate on thMe whose work applies 
most directly to the development of this ^per^ At the same time, workers 
whose contributions preceded the reif ication of deciiion theory as a dis- ^ 
Qipliae or at least a point of view are Ineluded, 

The first important work to be considered is that of Taylor and Russell 
(1939). Their tables, ihowlng the proportion of selected employees who prove 
to be successful under various combinations of selection ratios and success 
ratlosi set the stege for modern psychometric decision theory, as set forth 
in Grontaeh and Gleier (1965). Stilly the Taylor-Rusiell tables are tables, 
not a restively simple procedure yielding a single index* 

The usefulness of a single index to evaluate a validity coefficient is 
ftiir^ straightforward. One should be able to ask, "How much greater a 
coefficient do I need in order to have a measure tlmt works twice as well 
as the measure I have at hand?" 

Two a^roaches to the single-number problem here designated as "decision- 
theoretic" have been adTOnaed^ The first that of Brogden (19U6) who 
demonstrated that 

r is the ratio of the increase obtained by seleeting above 
a given standard score on the predictor to the increase 
tlmt would be obtained by selecting above the saK\j standard 
score on the criterion itself (p, 68), 

when the marginal distrlbutioni of the two variables are identical* Not^, 

however^ that the emphasis wai on the scores obtained by those selectad, 

not Oh the accuracy ef the selection itself. 



The second index with a decision-theoretic orientation is that of 
Jenkins (1953). His "index of selective efficiency," S, ii equal to the ratio, 
(Successes - Failures )/(SuGcesses + failures), vhei-e "Sueeesses" are "hits,"- 
io dtcialon-theoretic terms, Jenkins did not report this, but if the splits 
are made at the medians of normally-distributed nmrginals or (equivalently) 
If there are diehotomous distributions with 50-50 splits then the tetra- 
chorio r or the phi coefficient that would result would equal S, 

The Index of Predictive Efficiency 
Sheppard's theorem on raedian dichotomy (Kendall and Stuart/ 1963) is 
that, for any value of r, the probability that a person who scores above 
the median on the predictor will also score above the median on the criterion 
is 

pO 1 arc a in r , ^ 

where are sin r is expressed in radians, assuming that the narginals ar- 
distributed normally and the Joint distribution is bivarlate normal. 

When r . 0, :^ - .25| when r = 1, p° = ,5 . (arc sin r)/2,r thus varies 
from 0 to .25 and represents the proportion of indlviduflls for whom selection 

placement) Is improved over chanee selection by making use of a predictor 
of validity r. This value, when divided by .25, the proportion for whom 
prediction could be improved, yields 

e - S ftrc Bin r 

^ (2) 

This index Is called the "Index of predictive efficiency." Its curve, 

■ . .g , , . _ _ 

together with those of r" and E, is plotted in Figure 1. 

Examination of Figure 1 reveals certain intereBting features of e. 

First, it is equal to r at r « 0 and r ^ tl. Further, unlike functions 
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of T 9 it ne^tive \^hen r is negative. Its absolute value is, except 
St r = tl or r ?e Oj alv^ys ^eater tlmn that of E* Its absolute value is 

a 

greater than that of r from r m -*707 to r m *707. It must be noted. 

2 2 

here that arc sin *707 - ^5° * e = r ^ k" at that pointy -where the 
veetors representing the predictor and drlterion are at Ug^ „ That angle 
represent i a correlation halfway between zero and unity « Finally, between 
p = -*707 B.nd r m s707j the curve of e is approxlmtely linear^ 

An informal empirical check of e irag conducted, using one hundred 
"eD^minees" and four variables. The 'mriables were selected at random to 
have first essentially zero correlation and then progreisively higher ones, 
approaching. , unity 9 The original form of the marginals m^B approxiiwtely 
rectangular^ They then underwent tranaforimtlons to both approximate 
norMtllty and marked ne^tlve skewness. The obtained values of r mnged 
from -.03 to *97, 

The results ware frustrating. Somehow, although the "unrelated^^ 
variables had r's of -03, ,01^ and -.03^ for the rectangular, normal, and 
ikewed casei, respectively, they had obierved e's of approximately *2 
for the .25^ *5> and ^75 slectlon ratios^ However^ the plot of the visual 
Qentroids of the variqus TOluea of the observed e's followed that of 
Equation (2) « Moreover, there appeared to be no syitematlc relatlonshipi 
among nagnitude of r, shape of narglnal, and e* 

It must be concluded that e needi further itudy. Its propertlei are 
provocative* As an Instance, that it is approxlMitely linear in the range 
of validity coefficients that are reasonable for mental teste sheds new 
light on the Brodgen and Jenkins approaches^ so long in the dark. . The 
most obvious direction of this further study Is an extenilve exploration of 
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emplrically-derived mlues of a, with varied marginal dlstributlong, bi- 
vari&te dietributions, and selection ratios. 
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F(»TNOTE 

1 . ■ 

The use of "predictor" and "criterion'' in this paper 
are for convenience and are not intended ta limit 
the generality of the principles contained herein. 
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